AutoGPU : Automatic Generation of CUDA Kernel Code

Manual optimization of a CUDA kernel can be an arduous task, even for the simplest of kernels. The CUDA programming model is such that a high performance may only be achieved if memory accesses in the kernel follow certain patterns; further, fine-tuning of the kernel execution and loop configuration may result in a dramatic increase in performance. The number of possible such configurations make it particularly difficult to obtain a truly fast implementation of a given kernel. To that end, the AutoGPU program was developed: A tool that can measure the performance of a kernel for a defined set of configurations in order to decide on the optimal one. Moreover, the AutoGPU program can operate on a serial version of a desired kernel, and apply a series of transformations to change the memory access pattern of the kernel, besides deducing the optimal configuration for it. Hence, a CUDA kernel is automatically generated that may achieve a very high performance, without requiring that the user has knowledge of the CUDA programming model. Of course, tweaking of the generated kernel may still be applicable, but it is expected that it will be minor. The AutoGPU program was developed in the Standard ML language, and this report presents its structure and capabilities.